1. Field of the Invention
The present invention relates to a three-dimensional shape measurement technique and in particular to a method and apparatus for measuring a three-dimensional shape based on a plurality of three-dimensional shapes measured in different directions.
2. Description of the Related Art
There have been proposed a variety of techniques for measuring a three-dimensional shape of an object. As one of such techniques, a sinusoidal pattern projection method, which measuring a three-dimensional shape by projecting a sinusoidal pattern, will be described hereinafter based on Publication (1) entitled: “Grating Projection System for Profiling with the Aid of Fringe Scanning Method” Journal of Precision Engineering (JSPE), vol. 55, No. 10, pp. 1817/1822, 1989.
FIG. 1 shows a schematic diagram of a three-dimensional shape measurement system disclosed in the Publication (1). Referring to FIG. 1, a light pattern having a sinusoidal luminance pattern is projected from a light source 101 on an object 100 through a sinusoidal grating 102 having a gray scale values printed thereon sinusoidally. A point 104 on the object 100 is scanned by a camera 103 to produce image data. Assuming that a coordinate value of the point 104 on an image captured by the camera 103 is denoted by x and a luminance value on the coordinate x is denoted by I(x), the luminance value I(x) is given by the following equation (1):I(x)−A0(x)+A(x)cos(φ+α(x))  (1),where A0(x) is a bias component, and φ and α (x) each denote phases.
The image is captured by the camera 103, each time after shifting the grating 102 by a length equal to 1/N of the wavelength of a printed sinusoidal pattern N times, along an axis u, with the object 100 remaining stationary. The captured image appears as if the sinusoidal light pattern projected on the object 100 is proceeding 2π/N radian each time. Assuming that the phase φ is shifted from 0 radian up to 2(N−1) π/N radian where N is a positive integer, with an increment of 2π/N radian each time, the luminance value Ik(x) at a point x obtained for the k-th shifting (0≦k<N) is given by the following equation (2):Ik(x)−A0(x)+A(x)cos(2πk/N+α(x))  (2).
The phase α(x) is a phase value at a point x in an image captured at k=0. The point 104 is present on a half-line originating from the coordinate x on the camera screen to pass through a lens center. The point 104 as viewed from the light source 101 is present on a plane of the sinusoidal grating 102 determined by a straight line with the phase α(x) and the light source 101. Therefore, if a point of intersection between the straight line and the plane is found, it may be seen that the three-dimensional coordinate value of the point of intersection is that of the point 104.
By introducing two new coefficients A1(x) and B1(x) as shown by the following equation (3), the above equation (2) can be rewritten to the following equation (4).A1(x)=A(x)cosα(x), B1(x)=−A(x)sinα(x)  (3)Ik(x)=A0(x)+A1(x)cos2πk/N+B1(x)sin2πk/N  (4).A1(x) and B1(x) may be found by a following equation (5), using luminance values I0(x), . . . IN−1(x) at the point x obtained by N image-capturing operations:
                                                                                          A                  1                                ⁡                                  (                  x                  )                                            =                            ⁢                                                1                  N                                ⁢                                                      ∑                                          k                      =                      0                                                              N                      ⁢                                                                                          ⁢                      1                                                        ⁢                                                                          ⁢                                                            I                      k                                        ⁢                                                                  cos                        ⁡                                                  (                                                                                    2                              ⁢                              π                              ⁢                                                                                                                          ⁢                              k                                                        N                                                    )                                                                    .                                                                                                                                                                                B                  1                                ⁡                                  (                  x                  )                                            =                            ⁢                                                1                  N                                ⁢                                                      ∑                                          k                      ⁢                                                                                          ⁢                      0                                                              N                      -                      1                                                        ⁢                                                                          ⁢                                                            I                      k                                        ⁢                                          sin                      ⁡                                              (                                                                              2                            ⁢                            π                            ⁢                                                                                                                  ⁢                            k                                                    N                                                )                                                                                                                                                    (        5        )            
The phase α(x) may be found by the following equation (6):α(x)=tan−1(−B1(x)/A1(x))  (6).
The phase value of the object 100 on the image is obtained by executing the above-described phase calculations for each pixel on the image taken by the camera 103.
Meanwhile, the phase α (x) obtained from the above equation (6) is unexceptionally wrapped (folded) between −π and π, as may be seen from the fact that calculation is made using an arctangent function tan−1 ( ). The result is that the phase α(x) as found exhibits indefiniteness corresponding to an integer number times 2π, such that, in this state, a three-dimensional shape of the object 100 cannot be found.
By projecting a sinusoidal pattern composed of a period on the entire object 100, the phase α(x) can be uniquely determined. Since, a narrow phase value from −π to π is allocated at this time to the entire object 100, a high measurement accuracy cannot be achieved.
For this reason, a method is adopted to improve, a measurement accuracy at the cost of phase uniqueness in which a domain of an initial phase is enlarged and a sinusoidal pattern of plural periods is projected on the object 100.
FIG. 2 shows a phase image 105 which is an example of the phase as found in each pixel within an image date captured by the camera 103. In FIG. 2, a phase taking a value from −π to π is allocated to black to white.
If, for example, a plane is measured, as shown in FIG. 2, there is obtained non-continuous phase, so that it is necessary to determine a relative phase value within an image by suitable techniques to convert non-continuous phases into continuous values. Also, since an absolute phase value cannot be directly obtained, the absolute phase needs to be determined by some means or other.
To solve the above problem, a three-dimensional shape measurement technique has been proposed in U.S. Pat. No. 6,421,629.
FIG. 3 illustrates a state of light projection and a state of an object and cameras in an example of the three-dimensional shape measurement technique disclosed in the U.S. Pat. No. 6,421,629.
As shown in FIG. 3, a light projector-A 201 and a light projector-B 202 project light patterns having spatially sinusoidal luminance distribution onto the object 100. A camera-A 203 and a camera-B 204 capture images of the object 100 in different lines of vision. The conventional three-dimensional shape measurement technique measures the 3-dimensional shape of the object 100 based on the 3-dimensional shape of the object 100 measured by a pair of the projector-A 201 and the camera-A 203 and the 3-dimensional stripe of the object 100 measured by a pair of the projector-B 202 and the camera-B 204.
FIG. 4 illustrates a basic principle of absolute phase determination as disclosed in the U.S. Pat. No. 6,421,629. FIG. 4(A) shows an image-A (light pattern A) 116, which is a phase image obtained from an image captured by the camera-A 203 with the light projector-A 201 projecting the sinusoidal pattern. FIG. 4(B) shows an image-B (light pattern A) 117, which is a phase image obtained from an image captured by the camera-B 204 with the light projector-A 201 projecting the sinusoidal pattern. The absolute phase of a point-P 112 on the image A 116, that is, a 3-dimensional coordinate position, can be obtained by the following procedure.
First, as shown in FIG. 4(C), the candidate positions of the point P on the 3-dimensional space are obtained as points P1 113, P2 114 and P3 115, from the phase value at the point-P 112 on the image-A (light pattern A) 116 taking into account in determination among integral multiples of 2π. Subsequently, the lines of visions from the camera B 204 to respective ones of the points P1 113, P2 114 and P3 115 are obtained and are plotted on the image-B (light pattern A) 117 as shown in FIG. 4(B). Points which are viewed from the camera-B 204, each corresponding to the points P1 113, P2 114 and P3 115, can be obtained from phase values of the corresponding points on the image B (light pattern A) 117. Accordingly, it is considered that the 3-dimensional coordinate is properly measured and the absolute phase value thereof can be uniquely determined when the relative phase value of only one of the candidate points on the image-B (light pattern A) 117 matches that of the point P 112. By repeatedly performing the above procedure for all points on the image-A (light pattern A) 116, the absolute phase data of a plurality of points on the image-A (light pattern A) 116 can be obtained. Similarly, the absolute phase data of a plurality of points on the image-B (light pattern A) 117 can be obtained.
A more probable absolute phase can be obtained by the following procedure: calculating a product of pixel positions at which the absolute phase data are obtained on the image-A (light pattern A) 116 and the image-B (light pattern A) 117; and masking absolute phase data such that only the absolute phase data of a pixel position at which the calculated product exists is outputted.
The above-described procedure does not determine the absolute phases for all positions but for a great majority of positions in the region viewed by both the camera-A 203 and the camera-B 204. However, the absolute phases of the whole image can be obtained by repeating the step of adding or subtracting an integral multiple of 2π so as to minimize the phase difference for each pixel around each of the positions at which the absolute phases have been obtained by the above procedure.
A first 3-dimensional shape can be obtained using the well-known principle of triangulation of the projector-A 201 and the camera-A 203 from the absolute phase data of the image-A (light pattern) 116. Similarly, a second 3-dimensional shape can be obtained using the triangulation of the projector-B 202 and the camera-B 204 from the absolute phase data of the image-B (light pattern A) 117. These first and second 3-dimensional shapes are synthesized to finally form the 3-dimensional shape of the whole object 100.
In the above-described conventional method, an ordinary camera may be used as the camera-A 203 or camera-B 204. Accordingly, A lighting device such as a flash can be used to capture a texture image which represents the appearance of a 3-dimensionally shaped object. The texture data is attached to the measured shape data allowing application to catalogues or the like, which allows the seeing of the object from various points of view by using the texture mapping technique in 3-dimensional computer graphics.
The above-described conventional 3-dimensional shape measurement technique can produce the perfect 3-dimensional shape data on the precondition that the position relationship between any of the projector-A 201 and the projector-B 202 and any of the camera-A 203 and the camera-B 204, are precisely determined. In other words, when any pair of them provides the approximately same coordinate position, the 3-dimensional shape of the whole object 100 can be obtained by just synthesizing the first and second 3-dimensional shapes, for example, complementary synthesis.
However, it is actually impossible to obtain the perfect posit-ion relationship among them, resulting in a slightly difference between shapes measured by any two pairs of them. Accordingly, when the measured shapes are simply synthesized, a discontinuous step is developed in the synthesized 3-dimensional shape of the object as shown in FIG. 5.
Referring to FIG. 5, the above-described 3-dimensional measurement is performed in such an arrangement that the projector-A 201 and the camera-A 203 are placed at the left front of the object 100 and the projector-B 202 and the camera B 204 are placed at the right front of the object 100. A measured object 106 is a shape measured based on the visible range of the camera-A 204 on condition of the light pattern A and a measured object 107 is a shape measured based on the visible range of the camera-B 204 on condition of the light pattern B. As described above, it is difficult to exactly obtain the actual position relationship among the projectors and the cameras. Therefore, the measured objects 106 and 107 inevitably include some displacement or error in size. The simplest synthesis method is to divide the measured objects 106 and 107 by an appropriate plane to produce left-half portion 108 and right-half portion 109, which are complementary useful regions. In the simplest synthesis method, the left-half portion 108 and the right-half portion 109 are synthesized to reproduce the finally measured object 110 having discontinuous steps 111 such as displacement or error in size, resulting in reduced quality of 3-dimensional shape measurement.
There have been proposed several shape synthesizing methods. For example, Japanese Patent Application Unexamined Publication No. 5-303629 discloses a shape synthesizing method using both shape data and texture data for texture matching to synthesize the shapes into the shape of the whole object. However, in the case of an object having no pattern, this shape synthesizing method cannot be used. It is possible to manually change the relative position of the measured shapes to minimize the discontinuous steps 111. However, such adjustment must be performed for each shape measurement, expending much effort. In addition to the measured shapes, the obtained textures to be used for position matching inevitably include a slight discontinuous step because the characteristic of the camera-A 230 is generally different from that of the camera-B 204, resulting in reduced quality of texture data.
Japanese Patent Application Unexamined Publication No. 2001-084395 discloses a method of synthesizing two 3-dimensional data using the voxel (volume pixel) feature at a boundary region of the two shapes to gradually reduce a discontinuous step between the shapes. However, such a solution is not a fundamental solution to the above-described discontinuous step problem. Further, the boundary region must be adjusted on an ad-hoc basis.